On the influence of over-parameterization in manifold based surrogates and deep neural operators

Constructing accurate and generalizable approximators (surrogate models) for complex physico-chemical processes exhibiting highly non-smooth dynamics is challenging. The main question what type of surrogate models we should construct these be under-parameterized or over-parameterized. In this work, propose new developments perform comparisons two promising approaches: manifold-based polynomial chaos expansion (m-PCE) the deep neural operator (DeepONet), examine effect over-parameterization on generalization. While m-PCE enables construction a mapping by first identifying low-dimensional embeddings input functions, parameters, quantities interest (QoIs), learns nonlinear via use networks. We demonstrate performance methods in terms generalization accuracy solving 2D time-dependent Brusselator reaction-diffusion system with uncertainty sources, modeling an autocatalytic chemical reaction between species. extension constructing latent spaces formed separate functions output QoIs. To further enhance DeepONet, introduce weight self-adaptivity loss function. that DeepONet comparable cases relatively smooth input-output mappings. However, when considered, shows higher approximation accuracy. also find m-PCE, modest leads to better generalization, both within outside distribution, whereas aggressive over-fitting. contrast, even over-parameterized dynamics. Furthermore, compare above another recently proposed learning model, Fourier Neural Operator, show its Taken together, our studies can provide very good at low training cost, robustness noise but cost. methods, inference cost negligible.